-
Notifications
You must be signed in to change notification settings - Fork 38.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Automated cherry pick of #82152: Fix retry logic in DisruptionController #85460
Automated cherry pick of #82152: Fix retry logic in DisruptionController #85460
Conversation
This changes the retry logic in DisruptionController so that it reconciles update conflicts. In the old behavior, any pdb status update failure was retried with the same status, regardless of error. Now there is no retry logic with the status update. The error is passed up the stack where the PDB can be requeued for processing. If the PDB status update error is a conflict error, there are some new special cases: - failSafe is not triggered, since this is considered a retryable error - the PDB is requeued immediately (ignoring the rate limiter) because we assume that conflict can be resolved by getting the latest version
This tests the PDB status update path in DisruptionController and asserts that conflicting writes (with eviciton handler) are handled gracefully. This adds the client-go fake.Clientset into our tests, because that is the layer required for injecting update failures. This also adds a TestMain so that DisruptionController logs can be enabled during test. e.g., go test ./pkg/controller/disruption -v -args -v=4
@misterikkit: All 'parent' PRs of a cherry-pick PR must have one of the "release-note" or "release-note-action-required" labels, or this PR must follow the standard/parent release note labeling requirement. The following parent PRs have neither the "release-note" nor the "release-note-action-required" labels: #82152. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
@misterikkit: Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/release-note-none |
/approve This bug entirely defeats the purpose of PDB so it's important enough to backport. |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: lavalamp, misterikkit The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Cherry pick of #82152 on release-1.16.
#82152: Fix retry logic in DisruptionController
For details on the cherry pick process, see the cherry pick requests page.